Statistical benefits of choices from subsets

Marketers often analyze multinomial choice from a set of branded products to learn about demand. Given a set of brands to study, we analyze three reasons why choices from strict subsets of the brands can contain more statistical information about demand than choices from all the brands in the study: First, making choices from smaller subsets is easier, so it is possible to use more choice-tasks when the choice data comes from a choice-based conjoint survey. Second, choices from subsets of brands better identify and more accurately estimate the covariance structure of unobserved utility shocks associated with brands. Third, subsets automatically balance the brand-shares when some of the brands are less popular than others. We demonstrate these three “benefits of subsets” using a mixture of analytical results and numerical simulations, and provide implications for the design of choice-based conjoint analyses. We find that the optimal subset-size depends on the model, the number of brands in the study, and the designer’s resource constraint. Besides showing that subsets can be beneficial, we also provide a simulation methodology that helps designers pick the best subset-size for their setting. Contact: Robert Zeithammer, UCLA Anderson School of Management, 110 Westwood Plaza, Suite B408, Los Angeles, CA 90095-1481. email: [email protected] To learn about consumer demand, market researchers often analyze multinomial choice from a set of branded products. For example, the choice-based conjoint analysis technique analyzes consumer choices from different sets of hypothetical product-profiles described by their brands, prices, and other attributes (Louviere, Street and Burgess 2003). Given a fixed set of brands, a key survey-design question arises: How many brands should be included in each choice-task to maximize the demand information contained in the data? From a purely statistical point of view, i.e. assuming that the questionnaire respondents are well captured by standard random-utility models, it may seem that including all brands in each choice-set would always provide the most information about demand. In other words, excluding brands from choice-sets may seem like throwing away potentially useful data. We analyze statistical properties of standard choicemodels, and find that this intuition is incomplete: choices from random subsets of the considered brands can be statistically more informative compared to choices from all the brands. There are at least three reasons why a choice-based conjoint survey of branded products can produce more information about demand by offering respondents choices from random subsets of brands than by offering them choices from all brands under study. The most important reason turns out to also be the easiest to explain: choices from smaller sets are easier and faster, so the survey with smaller choice-sets can use more choice-tasks. Specifically, the decisiontheory literature (Bettman, Johnson and Payne 1990) implies that the ease and speed of making a choice are approximately linear in size of the choice set. This result implies that the designer can use more choice-tasks as long as the total number of profiles each respondent needs to process remains the same. Under this realistic constraint, we show both analytically for the multinomial logit model and by simulation for the multinomial probit model that randomized strict subsets allow sufficiently more tasks to improve demand estimation while keeping the survey time and difficulty perceived by subjects constant. Coupling smaller subsets with more choice-tasks per respondent is not necessary for subsets to be beneficial. We document two reasons why choices from random subsets of the considered brands can be statistically more informative compared to the same number of choices from all the brands: variance estimation and auto-balancing. The variance-estimation benefit is specific to models like the multinomial probit model (MNP), in which the covariance of the random utilities is estimated rather than assumed. Subsets can help because they provide exclusion restrictions. Specifically, the variance-covariance terms specific to brands absent from